Face model processing method and apparatus, non-volatile computer-readable storage medium, and electronic device

ABSTRACT

Embodiments of the present disclosure disclose a face model processing method performed at an electronic device. The method includes the following steps: obtaining a three-dimensional face model corresponding to a user picture, and selecting a sample oral cavity model in an oral cavity model library for the three-dimensional face model; registering the sample oral cavity model into the three-dimensional face model by using an oral cavity position parameter in the three-dimensional face model; performing form adjustment on an oral cavity form of the registered sample oral cavity model by using an expression parameter of the three-dimensional face model to generate a target oral cavity model; and generating, based on the three-dimensional face model and the target oral cavity model, a three-dimensional face expression model corresponding to the user picture.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of PCT Patent ApplicationNo. PCT/CN2019/077221, entitled “FACE MODEL PROCESSING METHOD ANDDEVICE, NONVOLATILE COMPUTER-READABLE STORAGE MEDIUM AND ELECTRONICDEVICE” filed on Mar. 6, 2019, which claims priority to Chinese PatentApplication No. 201810349627.6, entitled “FACE MODEL PROCESSING METHODAND DEVICE, AND STORAGE MEDIUM” filed Apr. 18, 2018, all of which areincorporated by reference in their entirety.

FIELD OF THE TECHNOLOGY

This application relates to the field of computer technologies, and inparticular, to a face model processing method and apparatus, anon-volatile computer-readable storage medium, and an electronic device.

BACKGROUND OF THE DISCLOSURE

Facial expression synthesis is one type of face synthesis technologies,is applied to the field of computer graphics such as movies, games,human-computer interaction and face recognition, and has a wideapplication prospect.

However, there often exists partial absence of an oral cavity when theoral cavity is in an opening form in a face picture synthesized by usingan existing face synthesis technology. To complement the oral cavitypart, a related technical solution mainly uses an oral cavitytwo-dimensional (2D) synthesis method and an oral cavity 3-dimensional(3D) synthesis method.

The oral cavity 2D synthesis method is first detecting a face region ofan original picture and obtaining an oral cavity region throughsegmentation, then searching a database for an oral cavity picture closeto a skin color of the face region, and filling the found oral cavitypicture in the oral cavity region. However, generally, it cannot beensured that the oral cavity picture obtained by using this method iscompletely consistent with an oral cavity form in the original picture,causing that a picture effect of a fused picture is poor. The oralcavity 3D synthesis method is first detecting feature points of a facein an original picture, then generating a 3D face model corresponding tothe face, and next directly fusing a 3D oral cavity model into the 3Dface model. However, because a size, lightness, an opening and closingform, and the like of the 3D oral cavity model are difficult to keepconsistent with those of the face model, the synthesized image isdifficult to achieve a relatively realistic effect in the oral cavity.Therefore, the oral cavity synthesis methods in the related art oftencause distortion, and have poor synthesis effects.

SUMMARY

According to various embodiments of this application, a face modelprocessing method and apparatus, a non-volatile computer-readablestorage medium, and an electronic device are provided.

A face model processing method, which is executed by an electronicdevice having a processor and memory storing a plurality of programs tobe executed by the processor, and may include:

obtaining a three-dimensional face model corresponding to a userpicture, and selecting a sample oral cavity model in an oral cavitymodel library for the three-dimensional face model;

registering the sample oral cavity model into the three-dimensional facemodel by using an oral cavity position parameter in thethree-dimensional face model;

performing form adjustment on an oral cavity form of the registeredsample oral cavity model by using an expression parameter of thethree-dimensional face model to generate a target oral cavity model; and

generating, based on the three-dimensional face model and the targetoral cavity model, a three-dimensional face expression modelcorresponding to the user picture.

A face model processing apparatus, which may include:

a model obtaining unit, configured to obtain a three-dimensional facemodel corresponding to a user picture, and obtain a selected sample oralcavity model in an oral cavity model library;

a model registration unit, configured to register the sample oral cavitymodel into the three-dimensional face model by using an oral cavityposition parameter in the three-dimensional face model;

a form adjustment unit, configured to perform form adjustment on an oralcavity form of the registered sample oral cavity model by using anexpression parameter of the three-dimensional face model to generate atarget oral cavity model; and

an expression model generating unit, configured to generate, based onthe three-dimensional face model and the target oral cavity model, athree-dimensional face expression model corresponding to the userpicture.

A non-transitory computer-readable storage medium, which stores aplurality of instructions, the instructions being configured to beloaded and executed by a processor to perform the foregoing methodoperations.

An electronic device, which may include a processor and a memory, thememory storing a computer program, the computer program being configuredto be loaded and executed by the processor to perform the followingoperations:

obtaining a three-dimensional face model corresponding to a userpicture, and selecting a sample oral cavity model in an oral cavitymodel library for the three-dimensional face model;

registering the sample oral cavity model into the three-dimensional facemodel by using an oral cavity position parameter in thethree-dimensional face model;

performing form adjustment on an oral cavity form of the registeredsample oral cavity model by using an expression parameter of thethree-dimensional face model to generate a target oral cavity model; and

generating, based on the three-dimensional face model and the targetoral cavity model, a three-dimensional face expression modelcorresponding to the user picture.

Details of one or more embodiments of this application are provided inthe accompany drawings and description below. Other features andadvantages of this application become more obvious from thespecification, the accompanying drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of thisapplication or in the related art more clearly, the following brieflydescribes the accompanying drawings required for describing theembodiments or the related art. Apparently, the accompanying drawings inthe following description show merely some embodiments of thisapplication, and a person of ordinary skill in the art may still deriveother drawings from the accompanying drawings without creative efforts.

FIG. 1a is an application scenario diagram of a face model processingmethod according to an embodiment of this application.

FIG. 1b is a schematic flowchart of a face model processing methodaccording to an embodiment of this application.

FIG. 2 is a schematic diagram of an interface of a three-dimensionalface model according to an embodiment of this application.

FIG. 3 is a schematic diagram of an interface of a sample oral cavitymodel according to an embodiment of this application.

FIG. 4 is a schematic diagram of displaying a model in a coordinatesystem according to an embodiment of this application.

FIG. 5 is a schematic diagram of an effect obtained after a sample oralcavity model is registered according to an embodiment of thisapplication.

FIG. 6 is a schematic flowchart of a face model processing methodaccording to an embodiment of this application.

FIG. 7 is a schematic diagram of an interface of a user pictureaccording to an embodiment of this application.

FIG. 8 is a schematic diagram of an interface of a three-dimensionalface model according to an embodiment of this application.

FIG. 9 is a schematic flowchart of a face model processing methodaccording to an embodiment of this application.

FIG. 10 is a schematic diagram of an effect obtained after a sample oralcavity model is registered according to an embodiment of thisapplication.

FIG. 11 is a schematic diagram of effects of oral cavity opening andclosing degrees in different opening and closing parameter conditionsaccording to an embodiment of this application.

FIG. 12 is a schematic flowchart of a face model processing methodaccording to an embodiment of this application.

FIG. 13 is a schematic diagram of an interface of feature pointselection positions according to an embodiment of this application.

FIG. 14 is a schematic diagram of an interface of an internal texture ofan oral cavity model according to an embodiment of this application.

FIG. 15 is a schematic structural diagram of a face model processingapparatus according to an embodiment of this application.

FIG. 16 is a schematic structural diagram of a face model processingapparatus according to an embodiment of this application.

FIG. 17 is a schematic structural diagram of a model obtaining unitaccording to an embodiment of this application.

FIG. 18 is a schematic structural diagram of a model registration unitaccording to an embodiment of this application.

FIG. 19 is a schematic structural diagram of a form adjustment unitaccording to an embodiment of this application.

FIG. 20 is a schematic structural diagram of a lightness adjustment unitaccording to an embodiment of this application.

FIG. 21 is a schematic structural diagram of a server according to anembodiment of this application.

DESCRIPTION OF EMBODIMENTS

The following clearly and completely describes the technical solutionsin the embodiments of this application with reference to theaccompanying drawings in the embodiments of this application.Apparently, the described embodiments are some of the embodiments ofthis application rather than all of the embodiments. Based on theembodiments of this application, all other embodiments obtained by aperson skilled in the art without paying any creative efforts all fallwithin the protection scope of the application.

FIG. 1a is a diagram of an application environment of a face modelprocessing method in an embodiment. As shown in FIG. 1a , a userterminal 110 is connected to a server 120 through a network. The facemodel processing method may be implemented by the terminal 110, or maybe implemented by the server 120, or may be collaboratively implementedby the terminal 110 and the server 120. The terminal 110 mayspecifically include terminal devices having an image processingfunction and an interaction function, such as a tablet computer, apersonal computer (PC), a smartphone, a palmtop computer, and a mobileInternet device (MID). The server 120 may be a server having an imageprocessing function, and may be implemented by using an independentserver or a server cluster formed by a plurality of servers.

The following describes the face model processing method provided in theembodiments of this application in detail with reference to FIG. 1b toFIG. 14.

FIG. 1 is a schematic flowchart of a face model processing methodaccording to an embodiment of this application. As shown in FIG. 1, themethod in this embodiment of this application may include the followingS101 to S104.

S101. Obtain a three-dimensional face model corresponding to a userpicture, and select a sample oral cavity model in an oral cavity modellibrary for the three-dimensional face model.

It may be understood that, the user picture is a picture that isselected by a user and that is used for establishing a three-dimensionalface model, may be a picture selected in a picture library, or may be apicture currently shot by a camera. The camera may be a fixed camera, ormay be a rotatable camera. The user picture includes a face image. Theremay be one or more face images. Certainly, the user picture mayalternatively be a face-like picture, such as a picture in a form ofsketch, ink and wash painting, animation, architecture, sculpture,artwork, or the like.

The obtaining a three-dimensional face model corresponding to a userpicture may be understood as that the electronic device obtains aselected user picture, performs face recognition processing on the userpicture to obtain the face image in the user picture, obtains athree-dimensional expression model in an expression model library, andsynthesizes the face image with the three-dimensional expression modelto generate the three-dimensional face model corresponding to the userpicture. The electronic device may be the user terminal 110 in FIG. 1,including terminal devices having an image processing function and aninteraction function such as a tablet computer, a PC, a smartphone, apalmtop computer, and an MID. In this case, the obtained user picture isread and obtained by the electronic device after being selected by theuser through the electronic device. The electronic device mayalternatively be the server 120 having an image processing function inFIG. 1. In this case, the obtained user picture is transmitted to theelectronic device by the user terminal after the user inputs a requestthrough the electronic device and is read and obtained by the electronicdevice.

The face recognition processing may be performing face detection on theuser picture. When the face image is detected, the electronic device maymark the detected face image, perform facial feature positioning or thelike. The face detection may be specifically performed according toactual scenario requirements. The face detection process may beimplemented by using face recognition methods, such as a facerecognition method based on feature face principal component analysis, aface recognition method of elastic graph matching, a face recognitionmethod of a support vector machine, and a face recognition method of adeep neural network.

The obtaining a three-dimensional expression model in an expressionmodel library may be randomly selecting a three-dimensional expressionmodel from an expression model set updated based on a current event(such as a current date, a hotspot event occurring on a current date, ora favorite theme selected by the user); or may be obtaining themeinformation of the user picture after analyzing the user picture, andsearching the expression model library for a three-dimensionalexpression model matching the theme information. The theme informationmay be described by using a keyword. For example, the theme informationmay be “Girl's Day”, “red packet”, or “happy”.

The obtaining a three-dimensional face model corresponding to a userpicture may also be understood as performing three-dimensionalreconstruction processing on the face image in the user picture by usinga three-dimensional reconstruction technology, and replacing anexpression parameter of the face model generated through thethree-dimensional reconstruction with an expression parameter selectedby the user to generate the three-dimensional face model. Thethree-dimensional reconstruction technology means that under a conditionthat depth information of a target object is known, three-dimensionalreconstruction of the target object may be implemented only throughregistration and fusion of point cloud data. Currently, thethree-dimensional reconstruction technology is classified into a passivethree-dimensional reconstruction technology and an activethree-dimensional reconstruction technology based on an obtaining methodfor the depth information of the target object. The passivethree-dimensional reconstruction technology generally uses an ambientenvironment such as reflection of natural light, uses a camera to obtainthe image, and then obtains three-dimensional spatial information of thetarget object by using a specific algorithm, and mainly includes atexture restoration shape method, a shadow restoration shape method anda stereo vision method. The active three-dimensional reconstructiontechnology means that a light source or an energy source, such as alaser, a sound wave or an electromagnetic wave is transmitted to thetarget object, and the depth information of the target object isobtained by receiving a returned light wave, and mainly includes a Moirefringe method, a time-of-flight method, a structured light method and atriangulation method.

The obtained three-dimensional face model may be described by using aformula S=S+Σw_(i)U_(i)+Σv_(i)V_(i)S=S+Σw_(i)U_(i)+Σv_(i)V_(i), where Sis a three-dimensional face model, S is a three-dimensional model of anaverage face, U_(i) and V_(i) are respectively a spatial basis vectormatrix of a face identity and a spatial basis vector matrix of a faceexpression obtained by training a face three-dimensional model data set,and w_(i) and v_(i) represent an identity parameter and an expressionparameter of a corresponding three-dimensional human model. S, U_(i),and V_(i) are known numbers. If w_(i) and v_(i) are known, acorresponding three-dimensional face model S may be calculated accordingto the foregoing formula. Correspondingly, if the three-dimensional facemodel S is synthesized, w_(i) and v_(i) corresponding to thethree-dimensional face model may also be obtained through detection.Generally, a presentation form of the three-dimensional face model S ischanged by changing values of w_(i) and v_(i). The identity parameterw_(i) of the three-dimensional face model remains unchanged to controlthe change of v_(i), so that geometric models of the same face withdifferent expressions may be obtained. For example, FIG. 2 shows agenerated three-dimensional expression model.

The oral cavity model library includes at least one oral cavity modelis, and each oral cavity model may be described by a formulaT=T+αΔT₁+βΔT₂+γΔT₃ T=T+αΔT₁+βΔT₂+γΔT₃, except that α, β, and γ are ofdifferent values. T is an oral cavity model in a closed state. ΔT₁, ΔT₂and ΔT₃ are an oral cavity opening and closing offset, a tonguestretching offset, and a tongue swinging offset respectively. α, β, andγ are an oral cavity opening and closing parameter, a tongue stretchingparameter, and a tongue swinging parameter respectively. An oral cavityopening and closing form, a tongue stretching form, and a tongueswinging form may be controlled by changing the values of α, β, and γrespectively. An oral cavity model randomly selected from the oralcavity model library is used as a sample oral cavity model. The sampleoral cavity model is shown in FIG. 3.

S102. Register the sample oral cavity model into the three-dimensionalface model by using an oral cavity position parameter in thethree-dimensional face model.

It may be understood that, the oral cavity position parameter is usedfor indicating registering the sample oral cavity model into thethree-dimensional face model. The registration includes registration ofa size and a position of the sample oral cavity model, so that thesample oral cavity model matches the three-dimensional face model.

In a specific implementation, the electronic device obtains an rotationangle and a translation of the sample oral cavity model relative to thethree-dimensional face model, and obtains current coordinate informationof the sample oral cavity model; obtains target coordinate informationof the sample oral cavity model in the three-dimensional face modelbased on the rotation angle, the translation, the current coordinateinformation, coordinate information of the three-dimensional face model,and coordinate information of an average face model; and moves thesample oral cavity model to a target position indicated by the targetcoordinate information. The rotation angle refers to a rotation angleand a rotation direction of the sample oral cavity model relative to thethree-dimensional face model in a space (a three-dimensional coordinatesystem). The rotation direction includes a pitching direction, ahorizontal deflection direction, and a spatial rolling direction. Therotation angle may be represented by a rotation matrix. The translationrefers to a relative distance between the sample oral cavity model andthe three-dimensional face model on a plane (such as projected onto an xor y plane), and may be obtained through calculation based oncoordinates of the sample oral cavity model and the three-dimensionalface model. The average face model may refer to extracting facialfeatures from a certain quantity of common faces, and averagingaccording to measurement data, and is obtained through synthesisaccording to the averaged value.

The obtaining an rotation angle and a translation of the sample oralcavity model relative to the three-dimensional face model may beunderstood as obtaining a rotation matrix R1 and a coordinate t1 of thesample oral cavity model in the same coordinate system, and obtaining arotation matrix R2 and a coordinate t2 of the three-dimensional facemodel. In this case, the rotation angle R of the sample oral cavitymodel relative to the three-dimensional face model may be obtainedthrough calculation based on R1 and R2, and the translation t may beobtained through calculation based on t1 and t2. Certainly, one of thesample oral cavity model and the three-dimensional face model in thesame coordinate system may alternatively be used as a reference model,and only the rotation angle R and the translation t of another modelrelative to the reference model need to be calculated. For example, asshown in FIG. 4, a three-dimensional face model S and a sample oralcavity model T are included. In the same coordinate system xyz, if S isused as a reference model, a rotation matrix of S relative to thecoordinate system is R1, and a rotation matrix of T relative to thecoordinate system is R2. In this case, an rotation angle R=R1/R2.Correspondingly, a translation of T relative to S is t=t0. It mayfurther be understood that, the three-dimensional expression modelstored in the expression model library and the oral cavity model storedin the oral cavity model library are frontally disposed (without angledeflection), and the generated three-dimensional face modelcorrespondingly rotates based on rotation of the face image in the userpicture. In this case, only the user picture needs to be analyzed toobtain a rotation matrix of the face image. The rotation matrix is anrotation angle R of the sample oral cavity model relative to thethree-dimensional face model. The translation t of the sample oralcavity model relative to the three-dimensional face model may still beobtained through calculation based on a position difference, or may beobtained by calculating a coordinate difference on the plane based onthe coordinate information of the three-dimensional face model and thecoordinate information of the sample oral cavity model.

In addition, because the size of the obtained sample oral cavity modeldoes not necessarily completely match a size of the three-dimensionalface model, the size of the sample oral cavity model needs to beadjusted according to the size of the three-dimensional face model (forexample, the size of the sample oral cavity model is adjusted accordingto a projection area of the three-dimensional face model projected ontothe x or y plane). A specific manner may be: obtaining an identityfeature parameter w_(i) of the face image in the user picture byanalyzing the user picture, and obtaining the coordinate information ofthe three-dimensional face model based on the identity feature parameterw_(i), an identity feature base U_(i), and the coordinate information ofthe average face model S; calculating a first area Area(π(S)) of thecoordinate information S of the three-dimensional face model projectedonto a plane (for example, the x or y plane) and a second areaArea(π(S))Area(π(S)) of the coordinate information S of the average facemodel projected onto the plane, and calculating a first ratio

$\frac{{Area}\mspace{14mu} \left( {\pi (S)} \right)}{{Area}\mspace{14mu} \left( {\pi \left( \overset{\_}{S} \right)} \right)}$

of the first area to the second area; and adjusting the size of thesample oral cavity model based on the first ratio, so that it can beensured that the size of the oral cavity model changes along with thesize of the three-dimensional face model.

During specific implementation, the obtained rotation angle R, thecurrent coordinate information T of the sample oral cavity model, thetranslation t, and the first ratio are substituted into a formula

$T^{*} = {{\frac{{Area}\mspace{14mu} \left( {\pi (S)} \right)}{{Area}\mspace{14mu} \left( {\pi \left( \overset{\_}{S} \right)} \right)} \cdot R \cdot T} + t}$

${T^{*} = {{\frac{{Area}\mspace{14mu} \left( {\pi (S)} \right)}{{Area}\mspace{14mu} \left( {\pi \left( \overset{\_}{S} \right)} \right)} \cdot R \cdot T} + t}},$

to obtain coordinate information T* of the sample oral cavity modelafter the adjustment of the size and the position. The transformationprocess from T to T* implements the fusion of the sample oral cavitymodel and the three-dimensional face model. FIG. 5 shows an effectdrawing after the fusion of S and T. The described coordinateinformation is the three-dimensional coordinate information.

S103. Perform form adjustment on an oral cavity form of the registeredsample oral cavity model by using the expression parameter of thethree-dimensional face model to generate a target oral cavity model.

It may be understood that, because the three-dimensional face model andthe sample oral cavity model have respective action mechanisms when theexpression changes, and are independent of each other, it is difficultto ensure that the oral cavity form of the sample oral cavity modelmatches the form of the three-dimensional face model. To ensurecoordination between the sample oral cavity model and thethree-dimensional face model, the oral cavity form of the sample oralcavity model needs to be adjusted.

The oral cavity form includes an oral cavity opening and closing form, atongue stretching form, and a tongue swinging form, which are controlledby values of an oral cavity opening and closing parameter α, a tonguestretching parameter β, and a tongue swinging parameter γ respectively.

The oral cavity opening and closing parameter α is obtained based on anexpression parameter v and a linkage weight η of the three-dimensionalface model. A specific implementation is as follows: obtaining aplurality of expression component parameters v_(i) corresponding to theexpression parameter v, separately calculating products of theexpression component parameters v_(i) in the plurality of expressioncomponent parameters and corresponding linkage weights η_(i), and thenperforming linear weighting processing on the products to obtain anopening and closing parameter α=Σ_(i∈G)η_(i)·v_(i)α=Σ_(i∈G)η_(i)·v_(i)of the oral cavity model, where G is a set of indexes i of theexpression parameter v_(i) related to oral cavity opening and closingactions in a formula S=S+Σw_(i)U_(i)+Σv_(i)V_(i)S=S+Σw_(i)U_(i)+Σv_(i)V_(i). A value of η_(i) is set by manuallyadjusting the oral cavity model and the three-dimensional face model, aparameter sample pair <α,v> is obtained when actions of the oral cavitymodel and the three-dimensional face model are coordinated, and then astatistical value of η is calculated by using a least square method toserve as an optimal prior parameter η_(i). A quantity of parametersample pairs is preferably greater than or equal to twice a quantity ofG, so as to improve accuracy of calculation. The least square method(also referred to as method of the least squares) is a mathematicaloptimization technology. The mathematical optimization technologysearches for a best function matching the data by minimizing a sum ofsquares of errors. The least square method may be used to easily obtainthe unknown data, and enables the sum of squares of errors between theobtained data and actual data to be minimized.

The tongue stretching parameter β and the tongue swinging parameter γmay be obtained based on the expression parameter v_(i) in an oralcavity expression parameter set. That is, the expression parameter v_(i)is in a correspondence with the tongue stretching parameter β and thetongue swinging parameter γ. When the expression parameter iscorrespondingly stored in a table form, β and γ may be directly obtainedby looking up the table. The tongue stretching parameter and the tongueswinging parameter may alternatively be randomly selected from a tonguestretching parameter set and a tongue swinging parameter set updatedbased on the current event. A predefined oral cavity expressionparameter set may alternatively be searched based on the themeinformation of the user picture for β and γ matching the themeinformation. The theme information may be described by a keyword. Forexample, the theme information may be “sad”, “laugh”, “happy”, or thelike.

During specific implementation, the electronic device separately adjuststhe oral cavity opening and closing parameter α, the tongue stretchingparameter β, and the tongue swinging parameter γ of the sample oralcavity model to generate an adjusted sample oral cavity model, namely,the target oral cavity model.

Optionally, when a face skin color lightness value does not match anoral cavity lightness value, the electronic device performs lightnessadjustment on an oral cavity lightness value of the sample oral cavitymodel after the form adjustment by using the face skin color lightnessvalue of the user picture or controlling an illumination direction. Aspecific manner may be: calculating a scaling factor s_(tooth_light) ofthe oral cavity lightness value based on the face skin color lightnessvalue, and using the scaling factor s_(tooth_light) to scale the oralcavity lightness value according to a corresponding proportion after anoral cavity texture RGB (red, green, blue) space is converted to an HSL(hue, saturation, lightness) space, and finally, then converting thecavity lightness value to the RGB space for display.

S104. Generate, based on the three-dimensional face model and the targetoral cavity model, a three-dimensional face expression modelcorresponding to the user picture.

It may be understood that, the generated three-dimensional faceexpression model is a result of fusing the three-dimensional face modeland the target oral cavity model. That is, sizes and positions of thetarget oral cavity model and the three-dimensional face model arematched, form expressions are consistent, and lightness is coordinated.In addition, the adjusted three-dimensional face model and the targetoral cavity model are rendered to generate the three-dimensional faceexpression model.

In this embodiment of this application, the three-dimensional face modelcorresponding to the user picture is obtained, and the selected sampleoral cavity model is obtained in the oral cavity model library. Then,the sample oral cavity model is registered into the three-dimensionalface model by using the oral cavity position parameter in thethree-dimensional face model, and form adjustment is performed on theoral cavity form of the registered sample oral cavity model by using theexpression parameter of the three-dimensional face model to generate thetarget oral cavity model. The three-dimensional face expression modelcorresponding to the user picture is obtained based on the obtainedthree-dimensional face model and the generated target oral cavity model.According to the oral cavity position parameter and the expressionparameter of the three-dimensional face model, the position and thescale of the sample oral cavity model are configured and the oral cavityshape is adjusted respectively, so that the sample oral cavity model cankeep consistent with the three-dimensional face model, the problem ofsynthesis distortion of the sample oral cavity model and thethree-dimensional face model is resolved, and further the synthesiseffect of the generated three-dimensional face expression model is good.

FIG. 6 is a schematic flowchart of a face model processing methodaccording to an embodiment of this application. As shown in FIG. 6, themethod in this embodiment of this application may include the followingstep S201 to step S215.

S201. Obtain a user picture, and perform face recognition processing onthe user picture to obtain a face image in the user picture.

For description of the user picture, reference may be made to theexplanation of the user picture in S101 of the foregoing embodiment. Forexample, as shown in FIG. 7, a user picture including a face image isobtained.

During specific implementation, an electronic device obtains the userpicture, and performs the face recognition processing on the userpicture to recognize the face image included in the user picture. Theface recognition processing may be performing face detection on the userpicture. When the face image is detected, the electronic device may markthe detected face image, perform facial feature positioning or the like.The face detection may be specifically performed according to actualscenario requirements. The face detection process may be implemented byusing face recognition methods, such as a face recognition method basedon feature face principal component analysis, a face recognition methodof elastic graph matching, a face recognition method of a support vectormachine, and a face recognition method of a deep neural network.

For description of the electronic device, reference may be made to theexplanation of the electronic device in S101 of the foregoingembodiment.

S202. Obtain a three-dimensional expression model in an expression modellibrary, and synthesize the face image with the three-dimensionalexpression model to generate a three-dimensional face modelcorresponding to the user picture.

It may be understood that, the expression model library includes atleast one expression model. The expression model is a facial featuremodel with expressions. The obtaining a three-dimensional expressionmodel in an expression model library may be randomly selecting athree-dimensional expression model from an expression model set updatedbased on a current event, where for example, ten expression models areupdated based on a hotspot event occurring on a current date, and theelectronic device may randomly select one expression model from the tenexpression models to synthesize the three-dimensional face model, or maybe obtaining theme information of the user picture after analyzing theuser picture or extracting theme information carried in a request, andsearching the expression model library for a three-dimensionalexpression model matching the theme information.

The obtained three-dimensional face model may be described by using aformula S=S+Σw_(i)U_(i)+Σv_(i)V_(i)S=S+Σw_(i)U_(i)+Σv_(i)V_(i). For thethree-dimensional face model and description of parameters therein,reference may be made to S101 in the foregoing embodiment.

The obtaining a three-dimensional face model corresponding to the userpicture may be understood as obtaining a three-dimensional expressionmodel in an expression model library, and synthesizing the face imagewith the three-dimensional expression model to generate thethree-dimensional face model corresponding to the user picture. Forexample, FIG. 8 shows a three-dimensional expression model synthesizedbased on the user picture shown in FIG. 7.

During specific implementation, the three-dimensional expression modelis obtained in the expression model library based on the themeinformation of the user picture or based on the current event. Theelectronic device synthesizes the recognized face image in the userpicture with the obtained three-dimensional expression model to generatethe three-dimensional face model corresponding to the user picture.

S203. Obtain a selected sample oral cavity model in the oral cavitymodel library.

It may be understood that, the oral cavity model library includes atleast one oral cavity model, and each oral cavity model may be describedby using a formula T=T+αΔT₁+βΔT₂+γΔT₃. For detailed description,reference may be made to the explanation of S101 in the foregoingembodiment.

S204. Obtain an rotation angle and a translation of the sample oralcavity model relative to the three-dimensional face model, and obtaincurrent coordinate information of the sample oral cavity model.

It may be understood that, for description of the rotation angle and thetranslation, reference may be made to the explanation of S102 in theforegoing embodiment. For the manner of obtaining the rotation angle andthe translation of the sample oral cavity model relative to thethree-dimensional face model in S204, reference may be made to theobtaining manner of S102 in the foregoing embodiments.

The current coordinate information of the sample oral cavity model is athree-dimensional coordinate value of feature points of the sample oralcavity model in a current coordinate system. Current coordinateinformation of a prestored sample oral cavity model may be directlyobtained in the oral cavity model library.

S205. Obtain target coordinate information of the sample oral cavitymodel in the three-dimensional face model based on the rotation angle,the translation, the current coordinate information, coordinateinformation of the three-dimensional face model, and coordinateinformation of an average face model.

All the coordinate information (such as the coordinate information ofthe three-dimensional face model, the coordinate information of theaverage face model, the current coordinate information, and the targetcoordinate information) is a three-dimensional space coordinate, and isin a one-to-one correspondence with feature points of the model. Thatis, the coordinate information of each model is a three-dimensionalcoordinate set of the feature points of the model.

In a specific implementation, as shown in FIG. 9, the obtaining targetcoordinate information of the sample oral cavity model in thethree-dimensional face model based on the rotation angle, thetranslation, the current coordinate information, coordinate informationof the three-dimensional face model, and coordinate information of anaverage face model may include the following steps:

S301. Obtain an identity feature parameter of the face image in the userpicture, and obtain the coordinate information of the three-dimensionalface model based on the identity feature parameter, the identity featurebase, and the coordinate information of the average face model.

The identity feature parameter w_(i) of the face image in the userpicture may be obtained by analyzing the user picture. In addition, theidentity parameter is still unchanged when the expression parameter is 0in the three-dimensional face modelS=S+Σw_(i)U_(i)+Σv_(i)V_(i)S=S+Σw_(i)U_(i)+Σv_(i)V_(i), and therefore,the obtained identity feature parameter w_(i), the identity feature baseU_(i), and the coordinate information S of the average face model may besubstituted into a formula S=S+Σw_(i)U_(i) to obtain the coordinateinformation S of the three-dimensional face model.

S302. Calculate a first area of the coordinate information of thethree-dimensional face model projected onto a plane and a second area ofthe coordinate information of the average face model projected onto theplane, and calculate a first ratio of the first area to the second area.

It may be understood that, because the size of the obtained sample oralcavity model does not necessarily completely match a size of thethree-dimensional face model, the size of the sample oral cavity modelneeds to be adjusted according to the size of the three-dimensional facemodel. For a calculating method of the first ratio, reference may bemade to the calculating manner of S102 in the foregoing embodiment.

S303. Obtain the target coordinate information of the sample oral cavitymodel in the three-dimensional face model based on the first ratio, therotation angle, the translation, and the current coordinate information.

During specific implementation, the electronic device substitutes theobtained rotation angle R, the current coordinate information T of thesample oral cavity model, the translation t, and the first ratio

$\frac{{Area}\mspace{14mu} \left( {\pi (S)} \right)}{{Area}\mspace{14mu} \left( {\pi \left( \overset{\_}{S} \right)} \right)}$

into a formula

${T^{*} = {{\frac{{Area}\mspace{14mu} \left( {\pi (S)} \right)}{{Area}\mspace{14mu} \left( {\pi \left( \overset{\_}{S} \right)} \right)} \cdot R \cdot T} + t}},$

to obtain the coordinate information T* of the sample oral cavity modelafter the adjustment of the size and the position. T* is the targetcoordinate information of the sample oral cavity model in thethree-dimensional face model.

S206. Move the sample oral cavity model to a target position indicatedby the target coordinate information.

The transformation process from T to T* implements the fusion of thesample oral cavity model and the three-dimensional face model. Forexample, FIG. 10 is a diagram of an effect obtained after the sampleoral cavity model is configured to the three-dimensional face modelshown in FIG. 8.

S207. Obtain the expression parameter of the three-dimensional facemodel, and obtain an oral cavity opening and closing parameter of thesample oral cavity model based on the expression parameter and a linkageweight.

For description of the oral cavity opening and closing parameter and thelinkage weight, reference may be made to S103 in the foregoingembodiment.

For example, a to d in FIG. 11 respectively describe an opening andclosing degree of the registered sample oral cavity model with differentα values.

S208. Obtain a tongue stretching parameter and a tongue swingingparameter indicated by the expression parameter in an oral cavityexpression parameter set.

The tongue stretching parameter β is used for controlling a stretchingdegree of a tongue, and the tongue swinging parameter γ is used forcontrolling a left-right swinging degree of the tongue. For an obtainingmanner of the tongue stretching parameter β and the tongue swingingparameter γ, reference may be made to S103 in the foregoing embodiment.

S209. Perform form adjustment on the oral cavity form of the sample oralcavity model based on the oral cavity opening and closing parameter, thetongue stretching parameter, and the tongue swinging parameter.

It may be understood that, because the three-dimensional face model andthe sample oral cavity model have respective action mechanisms when theexpression changes, and are independent of each other, it is difficultto ensure that the oral cavity form of the sample oral cavity modelmatches the form of the three-dimensional face model. To ensurecoordination between the sample oral cavity model and thethree-dimensional face model, the oral cavity form of the sample oralcavity model needs to be adjusted. That is, the oral cavity form of thesample oral cavity model registered into the three-dimensional facemodel is adjusted.

Specifically, if the registered sample oral cavity modelT*=T+α1ΔT₁+β1ΔT₂+γ1ΔT₃T*=T+α1ΔT₁+β1ΔT₂+γ1ΔT₂, the oral cavity openingand closing parameter α2 obtained in S207 and the tongue stretchingparameter β2 and the tongue swinging parameter γ2 obtained in S208replace α1, β1, and γ1, so as to adjust the oral cavity form of thesample oral cavity model.

S210. Perform lightness adjustment on an oral cavity lightness value ofthe sample oral cavity model after the form adjustment by using a faceskin color lightness value of the user picture to generate the targetoral cavity model.

It may be understood that, the lightness values of the exposed oralcavity model in different environments are different. For example, whenthe light is relatively dark, a facial skin color of the face model isusually quite dark. If the lightness of the oral cavity model is notadjusted but directly rendered, the displayed oral cavity model is quitebright but uncoordinated. Therefore, when the face skin color lightnessvalue does not match the oral cavity lightness value (for example, adifference between the face skin color lightness value and the oralcavity lightness value exceeds a specified threshold range), lightnessadjustment is performed on the oral cavity lightness value of the sampleoral cavity model after the form adjustment by using the face skin colorlightness value of the user picture.

In a specific implementation, as shown in FIG. 12, the performinglightness adjustment on an oral cavity lightness value of the sampleoral cavity model after the form adjustment by using a face skin colorlightness value of the user picture may include the following steps:

S401. Select a feature point set at a specified portion of thethree-dimensional face model.

It may be understood that, the specified portion may be a face foreheadand a cheek of the three-dimensional face model. At least one featurepoint is randomly selected at the specified portion respectively. The atleast one feature point may include a vertex. FIG. 13 shows a face imageof the three-dimensional face model, where A, B and C are the selectedfeature point set.

S402. Obtain a skin color lightness value of feature points in the userpicture and a skin color lightness value of the feature points in areference picture in the feature point set.

For example, if an A set includes ten feature points A1 to A10, a B setincludes ten feature points B1 to B10, and a C set includes ten featurepoints C1 to C10, a skin color (skin color texture) lightness valuel*_(p) of 30 feature points at a corresponding position of the userpicture is obtained, and linear weighting is performed on the obtainedskin color lightness value to obtain Σ_(p∈H) _(l*) _(p)Σ_(p∈H) _(l*)_(p). l*_(p) is a lightness value of the skin color texture at a pposition of the user picture, and H includes A, B and C. Similarly, theskin color (skin color texture) lightness value l_(p) of the 30 featurepoints at the corresponding position is obtained in the referencepicture (any user picture including a face image in a normal lightnesssituation) selected by a user, and the linear weighting is performed onthe obtained skin color lightness value to obtain Σ_(p∈H) _(l) _(p),l_(p) is texture lightness at a p position of the reference picture, andH includes A, B and C.

S403. Calculate a second ratio of the skin color lightness value of thefeature points in the user picture to the skin color lightness value ofthe feature points in the reference picture.

Specifically, the second ratio

$S_{tooth\_ light} = \frac{\Sigma_{p \in H}1_{p}^{*}\;}{\Sigma_{p \in H}1_{p}}$

is calculated based on the skin color lightness value obtained in S402.

The electronic device determines the lightness of the oral cavitytexture (as shown in FIG. 5) by calculating the ratio of lightness of anew face to the lightness of the reference face.

S404. Perform lightness adjustment on the oral cavity lightness value ofthe sample oral cavity model based on the second ratio.

Specifically, after the oral cavity texture of the sample oral cavitymodel is converted from an RGB space to an HSL space, the electronicdevice scales on the oral cavity lightness value of the oral cavitytexture at an equal proportion by using the second ratios_(tooth_light), and finally, converts the oral cavity lightness valueto the RGB space for display. An oral cavity texture image (each portionof the oral cavity) is shown in FIG. 14. The lightness value of thefeature points of the oral cavity texture image in FIG. 14 is adjustedbased on s_(tooth_light) obtained through calculation.

S211. Generate, based on the three-dimensional face model and the targetoral cavity model, a three-dimensional face expression modelcorresponding to the user picture.

It may be understood that, the generated three-dimensional faceexpression model is a result of fusing the three-dimensional face modeland the target oral cavity model. That is, sizes and positions of thetarget oral cavity model and the three-dimensional face model arematched, form expressions are consistent, and lightness is coordinated.In addition, the adjusted three-dimensional face model and the targetoral cavity model are rendered to generate the three-dimensional faceexpression model.

In a specific implementation scenario, this embodiment of thisapplication may further include the following steps.

S212. Convert the three-dimensional face expression model into anexpression picture corresponding to the user picture.

It may be understood that, the three-dimensional face expression modelis a three-dimensional image, and the expression picture is atwo-dimensional image. The conversion from three-dimensional faceexpression model to the expression picture is only conversion of a spacedimension, and picture content is unchanged.

S213. Add the expression picture to an expression picture sequencecorresponding to the user picture.

It may be understood that, the expression picture sequence may be formedby a plurality of frames of expression pictures. The expression picturesequence is a dynamic expression picture (for example, a GIF formatpicture) displayed according to a specified display sequence and a timeinterval. The adding the expression picture to an expression picturesequence corresponding to the user picture may be understood as creatingan empty expression picture sequence, and then the obtained plurality offrames of expression pictures are added to the empty expression picturesequence. It may further be understood that, an empty expression picturesequence is created, the time interval is set, and each time a frame ofexpression picture is generated, the frame of expression picture isadded to the empty expression picture sequence.

S214. Obtain, in a case of detecting that expression parameterscorresponding to two adjacent frames of expression pictures in theexpression picture sequence are discontinuous, continuous expressionparameters between the expression parameters corresponding to the twoadjacent frames of expression pictures.

It may be understood that, each frame of expression picture in theexpression picture sequence corresponds to one expression parameterrespectively, and the expression parameters of the two adjacent framesof expression pictures are continuous, so that when the expressionpicture sequence is displayed, the user may watch the expression picturesequence in which a set of expressions continuously change, which isinteresting. The face images corresponding to each frame of expressionpicture may be completely the same, or may be completely different, ormay be partially the same, which is not specifically limited.

During specific implementation, the electronic device traverses eachframe of expression picture, and reads a first expression parameter ofthe traversed current expression picture and a second expressionparameter of a next frame of expression picture. If the first expressionparameter and the second expression parameter are continuous, theelectronic device continuously reads a third expression parameter of anext frame of expression picture, and determines whether the secondexpression parameter and the third expression parameter are continuous.If the first expression parameter and the second expression parameterare discontinuous, the electronic device obtains continuous expressionparameters between the first expression parameter and the secondexpression parameter, and searches the picture library for an expressionpicture corresponding to the continuous expression parameters orgenerates an expression picture corresponding to the continuousexpression parameters by using the foregoing manner. There is at leastone continuous expression parameter. That is, at least one frame ofexpression picture needs to be inserted between two frames ofdiscontinuous expression pictures, so that the discontinuous expressionpictures become continuous expression pictures.

S215. Insert, between the two adjacent frames of expression pictures, anexpression picture corresponding to the continuous expressionparameters.

Specifically, the electronic device inserts the expression picturecorresponding to the obtained continuous expression parameters betweenthe two frames of discontinuous expression pictures, so that expressionscontinuously change.

Usually, a relatively large quantity of expression pictures need to beobtained to generate the expression picture sequence in which theexpressions continuously change. However, in order to reduce acalculation amount, the expression pictures with a specified quantity offrames may be first obtained, and then is supplemented by means ofinserting the frames (inserting the expression pictures).

In this embodiment of this application, the three-dimensional face modelcorresponding to the user picture is obtained, and the selected sampleoral cavity model is obtained in the oral cavity model library. Then,the sample oral cavity model is registered into the three-dimensionalface model by using the oral cavity position parameter in thethree-dimensional face model, and form adjustment is performed on theoral cavity form of the registered sample oral cavity model by using theexpression parameter of the three-dimensional face model to generate thetarget oral cavity model. The three-dimensional face expression modelcorresponding to the user picture is obtained based on the obtainedthree-dimensional face model and the generated target oral cavity model.According to the oral cavity position parameter and the expressionparameter of the three-dimensional face model, the position and thescale of the sample oral cavity model are configured and the oral cavityform is adjusted respectively, so that the sample oral cavity model cankeep consistent with the three-dimensional face model, the problem ofsynthesis distortion of the sample oral cavity model and thethree-dimensional face model is resolved, and further the synthesiseffect of the generated three-dimensional face expression model is good.

It may be understood that, although the steps in the flowcharts of FIG.1b , FIG. 6, FIG. 9 and FIG. 12 are sequentially shown according toindication of arrows, the steps are not necessarily performedsequentially according to the sequence indicated by the arrows. Unlessclearly specified in this specification, execution of the steps is notstrictly limited, and the steps may be performed in other sequences.Moreover, at least some of the steps in FIG. 1b , FIG. 6, FIG. 9 andFIG. 12 may include a plurality of substeps or a plurality of stages.The substeps or the stages are not necessarily performed at the samemoment, but may be performed at different moments. The substeps or thestages are not necessarily executed sequentially, but may be performedsequentially or alternately with other steps or at least some of thesubsteps or stages of other steps.

The following describes the face model processing apparatus provided inthe embodiments of this application in detail with reference to FIG. 15to FIG. 20. The apparatus shown in FIG. 15 to FIG. 20 are configured toperform the methods in the embodiments shown in FIG. 1 to FIG. 14 ofthis application. For ease of description, only a part related to theembodiments of this application is shown. For specific technical detailsthat are not disclosed, refer to the embodiments shown in FIG. 1 to FIG.14 of this application.

FIG. 15 is a schematic structural diagram of a face model processingapparatus according to an embodiment of this application. As shown inFIG. 15, the face model processing apparatus 1 in this embodiment ofthis application may include: a model obtaining unit 11, a modelregistration unit 12, a form adjustment unit 13, and an expression modelgenerating unit 14.

The model obtaining unit 11 is configured to obtain a three-dimensionalface model corresponding to a user picture, and obtain a selected sampleoral cavity model in an oral cavity model library.

The model registration unit 12 is configured to register the sample oralcavity model into the three-dimensional face model by using an oralcavity position parameter in the three-dimensional face model.

In a specific implementation, the model registration unit 12 isconfigured to: obtain an rotation angle and a translation of the sampleoral cavity model relative to the three-dimensional face model, andobtain current coordinate information of the sample oral cavity model;obtain target coordinate information of the sample oral cavity model inthe three-dimensional face model based on the rotation angle, thetranslation, the current coordinate information, coordinate informationof the three-dimensional face model, and coordinate information of anaverage face model; and move the sample oral cavity model to a targetposition indicated by the target coordinate information.

The form adjustment unit 13 is configured to perform form adjustment onan oral cavity form of the registered sample oral cavity model by usingan expression parameter of the three-dimensional face model to generatea target oral cavity model.

The expression model generating unit 14 is configured to generate, basedon the three-dimensional face model and the target oral cavity model, athree-dimensional face expression model corresponding to the userpicture.

For description of terms and manners such as the user picture, the oralcavity position parameter, the rotation angle, the three-dimensionalface model, the sample oral cavity model, and the adjustment manner oforal cavity lightness, reference may be made to the content in theforegoing face model processing method embodiments.

In this embodiment of this application, the three-dimensional face modelcorresponding to the user picture is obtained, and the selected sampleoral cavity model is obtained in the oral cavity model library. Then,the sample oral cavity model is registered into the three-dimensionalface model by using the oral cavity position parameter in thethree-dimensional face model, and form adjustment is performed on theoral cavity form of the registered sample oral cavity model by using theexpression parameter of the three-dimensional face model to generate thetarget oral cavity model. The three-dimensional face expression modelcorresponding to the user picture is obtained based on the obtainedthree-dimensional face model and the generated target oral cavity model.According to the oral cavity position parameter and the expressionparameter of the three-dimensional face model, the position and thescale of the sample oral cavity model are configured and the oral cavityform is adjusted respectively, so that the sample oral cavity model cankeep consistent with the three-dimensional face model, the problem ofsynthesis distortion of the sample oral cavity model and thethree-dimensional face model is resolved, and further the synthesiseffect of the generated three-dimensional face expression model is good.

FIG. 16 is a schematic structural diagram of a face model processingapparatus according to an embodiment of this application. As shown inFIG. 16, the face model processing apparatus 1 in this embodiment ofthis application may include: a model obtaining unit 11, a modelregistration unit 12, a form adjustment unit 13, an expression modelgenerating unit 14, a lightness adjustment unit 15, a model conversionunit 16, a picture adding unit 17, a picture detection unit 18, and apicture insertion unit 19.

The model obtaining unit 11 is configured to obtain a three-dimensionalface model corresponding to a user picture, and obtain a selected sampleoral cavity model in an oral cavity model library.

Optionally, as shown in FIG. 17, the model obtaining unit 11 includes:

a face image obtaining subunit 111, configured to obtain a selected userpicture, and perform face recognition processing on the user picture toobtain a face image in the user picture; and

a face model generating subunit 112, configured to obtain athree-dimensional expression model in an expression model library, andsynthesize the face image with the three-dimensional expression model togenerate the three-dimensional face model corresponding to the userpicture.

During specific implementation, the face model generating subunit 112 isconfigured to: obtain a three-dimensional expression model in anexpression model library based on theme information of the user pictureor based on a current event, and synthesize the recognized face image inthe user picture with the obtained three-dimensional expression model togenerate the three-dimensional face model corresponding to the userpicture.

The model registration unit 12 is configured to register the sample oralcavity model into the three-dimensional face model by using an oralcavity position parameter in the three-dimensional face model.

Optionally, as shown in FIG. 18, the model registration unit 12includes:

a parameter obtaining subunit 121, configured to obtain an rotationangle and a translation of the sample oral cavity model relative to thethree-dimensional face model, and obtain current coordinate informationof the sample oral cavity model, where

the current coordinate information of the sample oral cavity model is athree-dimensional coordinate value of feature points of the sample oralcavity model in a current coordinate system, and current coordinateinformation of a prestored sample oral cavity model may be directlyobtained in the oral cavity model library;

a target information obtaining subunit 122, configured to obtain targetcoordinate information of the sample oral cavity model in thethree-dimensional face model based on the rotation angle, thetranslation, the current coordinate information, coordinate informationof the three-dimensional face model, and coordinate information of anaverage face model; and

a model moving subunit 123, configured to move the sample oral cavitymodel to a target position indicated by the target coordinateinformation.

Optionally, the target information obtaining subunit 122 is specificallyconfigured to:

obtain an identity feature parameter of a face image in the userpicture, and obtain the coordinate information of the three-dimensionalface model based on the identity feature parameter, an identity featurebase, and the coordinate information of the average face model;

calculate a first area of the coordinate information of thethree-dimensional face model projected onto a plane and a second area ofthe coordinate information of the average face model projected onto theplane, and calculate a first ratio of the first area to the second area;and

obtain the target coordinate information of the sample oral cavity modelin the three-dimensional face model based on the first ratio, therotation angle, the translation, and the current coordinate information.

The form adjustment unit 13 is configured to perform form adjustment onan oral cavity form of the registered sample oral cavity model by usingan expression parameter of the three-dimensional face model to generatea target oral cavity model.

Optionally, as shown in FIG. 19, the form adjustment unit 13 includes:

a first parameter obtaining subunit 131, configured to obtain theexpression parameter of the three-dimensional face model, and obtain anoral cavity opening and closing parameter of the sample oral cavitymodel based on the expression parameter and a linkage weight;

a second parameter obtaining subunit 132, configured to obtain a tonguestretching parameter and a tongue swinging parameter indicated by theexpression parameter in an oral cavity expression parameter set; and

a form adjustment subunit 133, configured to perform form adjustment onthe oral cavity form of the sample oral cavity model based on the oralcavity opening and closing parameter, the tongue stretching parameter,and the tongue swinging parameter.

The lightness adjustment unit 15 is configured to perform lightnessadjustment on an oral cavity lightness value of the sample oral cavitymodel after the form adjustment by using a face skin color lightnessvalue of the user picture.

Optionally, as shown in FIG. 20, the lightness adjustment unit 15includes:

a feature point selection subunit 151, configured to select a featurepoint set at a specified portion of the three-dimensional face model;

a lightness value obtaining subunit 152, configured to obtain a skincolor lightness value of feature points in the user picture and a skincolor lightness value of the feature points in a reference picture inthe feature point set;

a second ratio obtaining subunit 153, configured to calculate a secondratio of the skin color lightness value of the feature points in theuser picture to the skin color lightness value of the feature points inthe reference picture; and

a lightness adjustment subunit 154, configured to perform lightnessadjustment on the oral cavity lightness value of the sample oral cavitymodel based on the second ratio.

The expression model generating unit 14 is configured to generate, basedon the three-dimensional face model and the target oral cavity model, athree-dimensional face expression model corresponding to the userpicture.

The model conversion unit 16 is configured to convert thethree-dimensional face expression model into an expression picturecorresponding to the user picture.

The picture adding unit 17 is configured to add the expression pictureto an expression picture sequence corresponding to the user picture.

The picture detection unit 18 is configured to obtain, in a case ofdetecting that expression parameters corresponding to two adjacentframes of expression pictures in the expression picture sequence arediscontinuous, continuous expression parameters between the expressionparameters corresponding to the two adjacent frames of expressionpictures.

The picture insertion unit 19 is configured to insert, between the twoadjacent frames of expression pictures, an expression picturecorresponding to the continuous expression parameters.

Specifically, the expression picture corresponding to the obtainedcontinuous expression parameters is inserted between two frames ofdiscontinuous expression pictures, so that expressions continuouslychange.

For terms and manners such as the three-dimensional face model, the userpicture, the expression model library, the oral cavity positionparameters, the rotation angles, the translation, the coordinateinformation, the calculating manner of the first ratio, the sample oralcavity model, and the adjustment manner of the oral cavity form,reference may be made to the content in the foregoing face modelprocessing method embodiments.

In this embodiment of this application, the three-dimensional face modelcorresponding to the user picture is obtained, and the selected sampleoral cavity model is obtained in the oral cavity model library. Then,the sample oral cavity model is registered into the three-dimensionalface model by using the oral cavity position parameter in thethree-dimensional face model, and form adjustment is performed on theoral cavity form of the registered sample oral cavity model by using theexpression parameter of the three-dimensional face model to generate thetarget oral cavity model. The three-dimensional face expression modelcorresponding to the user picture is obtained based on the obtainedthree-dimensional face model and the generated target oral cavity model.According to the oral cavity position parameter and the expressionparameter of the three-dimensional face model, the position and thescale of the sample oral cavity model are configured and the oral cavityform is adjusted respectively, so that the sample oral cavity model cankeep consistent with the three-dimensional face model, the problem ofsynthesis distortion of the sample oral cavity model and thethree-dimensional face model is resolved, and further the synthesiseffect of the generated three-dimensional face expression model is good.

An embodiment of this application further provides a non-volatilecomputer-readable storage medium. The non-volatile computer-readablestorage medium may store a plurality of instructions. The instructionsare configured to be loaded and executed by a processor to perform thesteps of the face model processing method in the embodiments shown inFIG. 1 to FIG. 14. For a specific execution process, reference may bemade to the specific description of the embodiments shown in FIG. 1 toFIG. 14.

FIG. 21 is a schematic structural diagram of an electronic deviceaccording to an embodiment of this application. As shown in FIG. 21, theelectronic device 1000 may include: at least one processor 1001 such asa central processing unit (CPU), at least one network interface 1004, auser interface 1003, a memory 1005, and at least one communications bus1002. The communications bus 1002 is configured to implement connectionand communication between the components. The user interface 1003 mayinclude a display and a camera. Optionally, the user interface 1003 mayfurther include a standard wired interface and wireless interface.Optionally, the network interface 1004 may include a standard wiredinterface and wireless interface (for example, a Wi-Fi interface). Thememory 1005 may be a high-speed RAM memory, or may be a non-volatilememory, for example, at least one magnetic disk memory. Optionally, thememory 1005 may further be at least one storage apparatus that islocated far away from the processor 1001. As shown in FIG. 21, thememory 1005, as a non-volatile computer-readable storage medium, mayinclude an operating system, a network communications module, a userinterface module, and a face model processing application program.

In the electronic device 1000 shown in FIG. 21, the user interface 1003is mainly configured to provide a user with an input interface andobtain data inputted by the user. The processor 1001 may be configuredto invoke a computer program stored in the memory 1005 for face modelprocessing. The computer program, when executed by the processor, causesthe processor to perform the steps of the face model processing methodin any of the foregoing embodiments.

A person of ordinary skill in the art may understand that all or some ofthe processes in the method of the foregoing embodiments may beimplemented by using the computer program to instruct related hardware.The program may be stored in a non-volatile computer-readable storagemedium. When the program is executed, the processes of the foregoingmethod embodiments may be performed. Any reference to the memory,storage, a database, or other media used in the embodiments provided inthis application may include a non-volatile and/or volatile memory. Thenon-volatile memory may include a read-only memory (ROM), a programmableROM (PROM), an electrically programmable ROM (EPROM), an electricallyerasable programmable ROM (EEPROM), or a flash. The volatile memory mayinclude a random access memory (RAM) or an external cache. As anillustration instead of a limitation, the RAM is available in variousforms, such as a static RAM (SRAM), a dynamic RAM (DRAM), a synchronousDRAM (SDRAM), a double data rate SDRAM (DDRSDRAM), an enhanced SDRAM(ESDRAM), a Synchlink DRAM (SLDRAM), a Rambus direct RAM (RDRAM), adirect rambus dynamic RAM (DRDRAM), and a rambus dynamic RAM (RDRAM).

The foregoing disclosure is merely exemplary embodiments of thisapplication, and certainly is not intended to limit the protection scopeof this application. Therefore, equivalent variations made in accordancewith the claims of this application shall fall within the scope of thisapplication.

What is claimed is:
 1. A face model processing method, executed by anelectronic device having a processor and memory storing a plurality ofprograms to be executed by the processor, the method comprising:obtaining a three-dimensional face model corresponding to a userpicture, and selecting a sample oral cavity model in an oral cavitymodel library for the three-dimensional face model; registering thesample oral cavity model into the three-dimensional face model by usingan oral cavity position parameter in the three-dimensional face model;performing form adjustment on an oral cavity form of the registeredsample oral cavity model by using an expression parameter of thethree-dimensional face model to generate a target oral cavity model; andgenerating, based on the three-dimensional face model and the targetoral cavity model, a three-dimensional face expression modelcorresponding to the user picture.
 2. The method according to claim 1,wherein the obtaining a three-dimensional face model corresponding to auser picture comprises: obtaining the user picture, and performing facerecognition processing on the user picture to obtain a face image in theuser picture; and obtaining a three-dimensional expression model in anexpression model library, and synthesizing the face image with thethree-dimensional expression model to generate the three-dimensionalface model corresponding to the user picture.
 3. The method according toclaim 1, wherein the registering the sample oral cavity model into thethree-dimensional face model by using an oral cavity position parameterin the three-dimensional face model comprises: obtaining a rotationangle and a translation of the sample oral cavity model relative to thethree-dimensional face model, and obtaining current coordinateinformation of the sample oral cavity model; obtaining target coordinateinformation of the sample oral cavity model in the three-dimensionalface model based on the rotation angle, the translation, the currentcoordinate information, coordinate information of the three-dimensionalface model, and coordinate information of an average face model; andmoving the sample oral cavity model to a target position indicated bythe target coordinate information.
 4. The method according to claim 3,wherein the obtaining target coordinate information of the sample oralcavity model in the three-dimensional face model based on the rotationangle, the translation, the current coordinate information, coordinateinformation of the three-dimensional face model, and coordinateinformation of an average face model comprises: obtaining an identityfeature parameter of a face image in the user picture, and obtaining thecoordinate information of the three-dimensional face model based on theidentity feature parameter, an identity feature base, and the coordinateinformation of the average face model; calculating a first area of thecoordinate information of the three-dimensional face model projectedonto a plane, calculating a second area of the coordinate information ofthe average face model projected onto the plane, and calculating a firstratio of the first area to the second area; and obtaining the targetcoordinate information of the sample oral cavity model in thethree-dimensional face model based on the first ratio, the rotationangle, the translation, and the current coordinate information.
 5. Themethod according to claim 1, wherein the performing form adjustment onan oral cavity form of the registered sample oral cavity model by usingan expression parameter of the three-dimensional face model comprises:obtaining the expression parameter of the three-dimensional face model,and obtaining an oral cavity opening and closing parameter of the sampleoral cavity model based on the expression parameter and a linkageweight; obtaining a tongue stretching parameter and a tongue swingingparameter indicated by the expression parameter in an oral cavityexpression parameter set; and performing form adjustment on the oralcavity form of the sample oral cavity model based on the oral cavityopening and closing parameter, the tongue stretching parameter, and thetongue swinging parameter.
 6. The method according to claim 1, whereinafter the performing form adjustment on an oral cavity form of theregistered sample oral cavity model by using an expression parameter ofthe three-dimensional face model, the method further comprises:performing lightness adjustment on an oral cavity lightness value of thesample oral cavity model after the form adjustment by using a face skincolor lightness value of the user picture.
 7. The method according toclaim 6, wherein the performing lightness adjustment on an oral cavitylightness value of the sample oral cavity model after the formadjustment by using a face skin color lightness value of the userpicture comprises: selecting a feature point set at a specified portionof the three-dimensional face model; obtaining a skin color lightnessvalue of feature points in the user picture and a skin color lightnessvalue of the feature points in a reference picture in the feature pointset; calculating a second ratio of the skin color lightness value of thefeature points in the user picture to the skin color lightness value ofthe feature points in the reference picture; and performing lightnessadjustment on the oral cavity lightness value of the sample oral cavitymodel based on the second ratio.
 8. The method according to claim 1,wherein after the generating, based on the three-dimensional face modeland the target oral cavity model, a three-dimensional face expressionmodel corresponding to the user picture, the method further comprises:converting the three-dimensional face expression model into anexpression picture corresponding to the user picture; adding theexpression picture to an expression picture sequence corresponding tothe user picture; obtaining, in a case of detecting that expressionparameters corresponding to two adjacent frames of expression picturesin the expression picture sequence are discontinuous, continuousexpression parameters between the expression parameters corresponding tothe two adjacent frames of expression pictures; and inserting, betweenthe two adjacent frames of expression pictures, an expression picturecorresponding to the continuous expression parameters.
 9. An electronicdevice, comprising a processor and a memory, the memory storing aplurality of computer programs, the computer programs, when executed bythe processor, causing the electronic device to perform a plurality ofoperations including: obtaining a three-dimensional face modelcorresponding to a user picture, and selecting a sample oral cavitymodel in an oral cavity model library for the three-dimensional facemodel; registering the sample oral cavity model into thethree-dimensional face model by using an oral cavity position parameterin the three-dimensional face model; performing form adjustment on anoral cavity form of the registered sample oral cavity model by using anexpression parameter of the three-dimensional face model to generate atarget oral cavity model; and generating, based on the three-dimensionalface model and the target oral cavity model, a three-dimensional faceexpression model corresponding to the user picture.
 10. The electronicdevice according to claim 9, wherein the obtaining a three-dimensionalface model corresponding to a user picture comprises: obtaining the userpicture, and performing face recognition processing on the user pictureto obtain a face image in the user picture; and obtaining athree-dimensional expression model in an expression model library, andsynthesizing the face image with the three-dimensional expression modelto generate the three-dimensional face model corresponding to the userpicture.
 11. The electronic device according to claim 9, wherein theregistering the sample oral cavity model into the three-dimensional facemodel by using an oral cavity position parameter in thethree-dimensional face model comprises: obtaining an rotation angle anda translation of the sample oral cavity model relative to thethree-dimensional face model, and obtaining current coordinateinformation of the sample oral cavity model; obtaining target coordinateinformation of the sample oral cavity model in the three-dimensionalface model based on the rotation angle, the translation, the currentcoordinate information, coordinate information of the three-dimensionalface model, and coordinate information of an average face model; andmoving the sample oral cavity model to a target position indicated bythe target coordinate information.
 12. The electronic device accordingto claim 11, wherein the obtaining target coordinate information of thesample oral cavity model in the three-dimensional face model based onthe rotation angle, the translation, the current coordinate information,coordinate information of the three-dimensional face model, andcoordinate information of an average face model comprises: obtaining anidentity feature parameter of a face image in the user picture, andobtaining the coordinate information of the three-dimensional face modelbased on the identity feature parameter, an identity feature base, andthe coordinate information of the average face model; calculating afirst area of the coordinate information of the three-dimensional facemodel projected onto a plane, calculating a second area of thecoordinate information of the average face model projected onto theplane, and calculating a first ratio of the first area to the secondarea; and obtaining the target coordinate information of the sample oralcavity model in the three-dimensional face model based on the firstratio, the rotation angle, the translation, and the current coordinateinformation.
 13. The electronic device according to claim 9, wherein theperforming form adjustment on an oral cavity form of the registeredsample oral cavity model by using an expression parameter of thethree-dimensional face model comprises: obtaining the expressionparameter of the three-dimensional face model, and obtaining an oralcavity opening and closing parameter of the sample oral cavity modelbased on the expression parameter and a linkage weight; obtaining atongue stretching parameter and a tongue swinging parameter indicated bythe expression parameter in an oral cavity expression parameter set; andperforming form adjustment on the oral cavity form of the sample oralcavity model based on the oral cavity opening and closing parameter, thetongue stretching parameter, and the tongue swinging parameter.
 14. Theelectronic device according to claim 9, wherein after the performingform adjustment on an oral cavity form of the registered sample oralcavity model by using an expression parameter of the three-dimensionalface model, the method further comprises: performing lightnessadjustment on an oral cavity lightness value of the sample oral cavitymodel after the form adjustment by using a face skin color lightnessvalue of the user picture.
 15. The electronic device according to claim14, wherein the performing lightness adjustment on an oral cavitylightness value of the sample oral cavity model after the formadjustment by using a face skin color lightness value of the userpicture comprises: selecting a feature point set at a specified portionof the three-dimensional face model; obtaining a skin color lightnessvalue of feature points in the user picture and a skin color lightnessvalue of the feature points in a reference picture in the feature pointset; calculating a second ratio of the skin color lightness value of thefeature points in the user picture to the skin color lightness value ofthe feature points in the reference picture; and performing lightnessadjustment on the oral cavity lightness value of the sample oral cavitymodel based on the second ratio.
 16. The electronic device according toclaim 9, wherein after the generating, based on the three-dimensionalface model and the target oral cavity model, a three-dimensional faceexpression model corresponding to the user picture, the method furthercomprises: converting the three-dimensional face expression model intoan expression picture corresponding to the user picture; adding theexpression picture to an expression picture sequence corresponding tothe user picture; obtaining, in a case of detecting that expressionparameters corresponding to two adjacent frames of expression picturesin the expression picture sequence are discontinuous, continuousexpression parameters between the expression parameters corresponding tothe two adjacent frames of expression pictures; and inserting, betweenthe two adjacent frames of expression pictures, an expression picturecorresponding to the continuous expression parameters.
 17. Anon-transitory computer-readable storage medium storing instructions,the instructions, when executed by a processor of an electronic device,cause the electronic device to perform a plurality of operationsincluding: obtaining a three-dimensional face model corresponding to auser picture, and selecting a sample oral cavity model in an oral cavitymodel library for the three-dimensional face model; registering thesample oral cavity model into the three-dimensional face model by usingan oral cavity position parameter in the three-dimensional face model;performing form adjustment on an oral cavity form of the registeredsample oral cavity model by using an expression parameter of thethree-dimensional face model to generate a target oral cavity model; andgenerating, based on the three-dimensional face model and the targetoral cavity model, a three-dimensional face expression modelcorresponding to the user picture.
 18. The non-transitorycomputer-readable storage medium according to claim 17, wherein theobtaining a three-dimensional face model corresponding to a user picturecomprises: obtaining the user picture, and performing face recognitionprocessing on the user picture to obtain a face image in the userpicture; and obtaining a three-dimensional expression model in anexpression model library, and synthesizing the face image with thethree-dimensional expression model to generate the three-dimensionalface model corresponding to the user picture.
 19. The non-transitorycomputer-readable storage medium according to claim 17, wherein theregistering the sample oral cavity model into the three-dimensional facemodel by using an oral cavity position parameter in thethree-dimensional face model comprises: obtaining an rotation angle anda translation of the sample oral cavity model relative to thethree-dimensional face model, and obtaining current coordinateinformation of the sample oral cavity model; obtaining target coordinateinformation of the sample oral cavity model in the three-dimensionalface model based on the rotation angle, the translation, the currentcoordinate information, coordinate information of the three-dimensionalface model, and coordinate information of an average face model; andmoving the sample oral cavity model to a target position indicated bythe target coordinate information.
 20. The non-transitorycomputer-readable storage medium according to claim 17, wherein theperforming form adjustment on an oral cavity form of the registeredsample oral cavity model by using an expression parameter of thethree-dimensional face model comprises: obtaining the expressionparameter of the three-dimensional face model, and obtaining an oralcavity opening and closing parameter of the sample oral cavity modelbased on the expression parameter and a linkage weight; obtaining atongue stretching parameter and a tongue swinging parameter indicated bythe expression parameter in an oral cavity expression parameter set; andperforming form adjustment on the oral cavity form of the sample oralcavity model based on the oral cavity opening and closing parameter, thetongue stretching parameter, and the tongue swinging parameter.